Goto

Collaborating Authors

 data mesh architecture


Leverage Machine Learning to Detect Insider Threats

#artificialintelligence

For an insider threat program to benefit from ML algorithms, first it must train and implement them. To succeed, machine learning algorithms must be trained against pre-collected, validated data sets. Collection, validation, and training all tend to be difficult and time-consuming. This is one of many areas where the benefits of data mesh come into play. Today, data collection happens continuously and at high volumes across a vast number of sources which must be governed and exposed to extract value.


The Data Mesh architecture

#artificialintelligence

The architecture of data is not just a technical architecture but is also an organizational structure, therefore, making it a key factor for building any data empire. Over time there have been introduced different types of architectures, always with the aim of covering the gaps with the ideal solution: we started with data warehouses which were mainly focused on creating structured datasets for reporting, then expanded to data lakes with the aim of having centralized access to the data wherever they are and in whichever form and to remove the pain points present in the data lake a new architecture called data mesh was introduced about 2 years ago by Zhamak Dehghani. We all know that this is a field full of buzzwords so whenever something new comes out it takes a while (if ever) to establish what it precisely means; the data mesh is not an exception. However, the simplest explanation is that it is a domain-oriented structuring of the data with a focus on data and data product ownership, driving towards a well-governed data usage as well as offering self-serve data infrastructure. But what do these keywords mean precisely?